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Abstract. In this paper we propose a new parallel algorithm for solving global optimization 
(GO) multidimensional problems. The method unifies two powerful approaches for accelerating 
the search : parallel computations and local tuning on the behavior of the objective function. 
We establish convergence conditions for the algorithm and theoretically show that the usage of 
local information during the global search permits to accelerate solving the problem significantly. 
Results of numerical experiments executed with 100 test functions arc also reported. 
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1. Introduction 

Let us consider the problem of finding the global minimum and global minimizers 
of a function (f)(y),y £ D, where 

D = {x : tti < X, < bi,l < i < N} 

is a hyperinterval in . This problem is actively studied by many authors (see 
e.g. [6], [7], [8], [9], [16]). One of the promising approaches to attack this problem 
is in using the Peano type space-filling curves. 

It is known (see [2], [16], [17]) that this multidimensional problem can be reduced 
to a one-dimensional one by using the curves. In this case we obtain 

min{0(y) : y E D} = mm{(p{y{x)) : x E [0, 1]}, (1) 

moreover, if the multidimensional function (fiijj) satisfies the Lipschitz condition 
with the constant L over D then (j){y{x)) over the interval [0, 1] satisfies the Holder 
condition 

|(/.(2/(x'))-0(y(x")) |<i/ x',x"E[0,l], (2) 

where H is the Holder constant and 

H = ALdVN, d = max{6, - at -.1 <i < N}. 

The constant H corresponds to the space- filling curve y{x) which computational 
scheme is presented in [16] (see also [18]). A special procedure for fast calculating 



the image y{x) G D on the curve for every given x G [0, 1] has been proposed in 
these papers. The procedure has a high speed because, instead of constructing an 
approximation of the whole curve, it directly computes the coordinates of y{x) in 
D for every given x. In addition, its parallel version has been introduced in [18]. 

For solving the problem (1), (2) the information algorithm (lA) has been proposed 
in [16], [17] for the case when the Lipschitz constant (and, therefore the Holder 
one) is unknown. A special procedure for estimating H on the base of information 
obtained during the search has been elaborated. 

The power of parallel computations is widely used in global optimization to accel- 
erate the search (different approaches are presented in [1], [5], [8] - [10], [14], [15], 
[18] [20]). It has been shown that very often simple parallelizing ideas (parallel 
grid method or subdividing the search region in p subregions where p processors 
work in parallel) can lead to appearence of redundant function evaluations in com- 
parison with an efficient sequential method. That is why one of the reasonable ways 
for usage of parallel computers consists in parallelizing fast sequential methods. 

Wide numerical experiments and a deep theoretical study have shown (see [4], 
[16]) that lA has good speed characteristics in comparison with other methods which 
don't use derivatives. In [18] a parallel information algorithm (PIA) extending lA 
to the case of parallel computations has been introduced. Convergence conditions 
and estimates of speed up obtained in comparison with the original sequential lA 
have been established. A special study has been executed to obtain conditions 
which guarantee absence of redundant (in comparison with the sequential case) 
evaluations of (j}{y{x)). This idea has demonstrated to be fruitful and it was applied 
to parallelize some other sequential GO methods (see [3], [14], [15]). By generalizing 
this approach a class of parallel characteristical GO algorithms has been introduced 
and theoretically investigated in [5]. 

An alternative approach for acceleration of the global search consists in the fol- 
lowing. It has been shown for different one-dimensional GO algorithms (see [11] 
- [13]) that using estimates of the Lipschitz constant L slows down the search for 
that subregions where the local Lipschitz constants are significantly less than L 
(hereinafter we shall call L the global Lipschitz constant). The sequential informa- 
tion GO algorithm with local tuning (lALT) has been introduced in [12] for solving 
the problem (1), (2). In that paper a special procedure has been proposed to esti- 
mate adaptively local constants in different areas of the search region to accelerate 
the search. Theoretical results and numerical experiments have shown that this ap- 
proach permits to accelerate the search significantly in comparison with the original 
lA using esimates of the global Lipschitz constant. 

In this paper we propose to unify both approaches and construct a parallel infor- 
mation algorithm with local tuning (PLT). In order to accelerate the search during 
every iteration the new method makes the following : 

- it adaptively estimates local Holder constants in different subintervals of [0, 1] to 
tune itself on the local behaviour of the reduced objective function (j){y{x)), x G 

[0,1]; 



- it simultaneously evaluates the objective fimetion (f>{y{x)) at p different points 
in p intervals (having the highest probability to contain the global optimum) 
on p parallel computers; 

- local information about <j){y{x)) is used over the whole search region during the 

global search in contrast with traditional approaches (see e.g. [6], [7]) usually 
starting a local search in a neighbourhood of a global minimizer after stopping 
the global procedure. 

It will be shown further that adaptive estimates of the local Holder constants 
introduced here permit to accelerate the search significantly in comparison with 
the parallel methods (see [5], [18]) using the global Lipschitz (Holder) constants or 
their estimates. 

The rest of the paper is structured as follows. Section 2 describes the compu- 
tational scheme of PLT and presents convergence results of the new type for the 
introduced parallel method. Section 3 contains results of a numerical comparison 
between lA, PIA, lALT and PLT executed with 100 two-dimensional functions 
usually used in literature for numerical comparison between the information GO 
methods (see [4], [5], [16], [18]). Some conclusions are presented in the last Section. 

2. The algorithm and its convergence conditions 

To describe the algorithm we need some designations and definitions. We introduce 
a simplifying designation 

f{x) = cj){y{x)), xe[0,l], 

for the objective function (j){y(x)). We call trial the operation of evaluating /(x) 
at a point x. The character I we use as the counter of parallel iterations of PLT. 
In the course of every parallel iteration I we evaluate f{x) at p{l) > 1 points using 
p{l) parallel processors. We use the Peano curve approximation with the depth m 
presented in [16], [18] and a preset accuracy e concorded with m by the inequality 

Let us now present the computational scheme of the new method. 

Step 0. Execute q{l) > 1 initial trials at the points = 0, = 1 and some 
internal points x"^, x'', . . . , x''^'-' belonging to (0, 1). Set I = 1. 

Step 1. Points ■^"^ of the previous trials reorder by increasing of their 

coordinates using subscripts, i.e. 

= Xl < X2 < . . . < Xg_l < Xg = 1 



where q = q{l). 



Step 2. Calculate the values fij estimating local Holder constants for the intervals 
],2 < j < q, following the rules : 

H = max{Aj, 7j-, 2 < j < q, (3) 

where ^ > is a small number, the value A^- spying on the local information is 
determened by the formulae 

Aj=ma^{ |i/;v :^e-fj}, (4) 



{2,3} ifi = 2 

Ij = { {J-1,J,J + 1} if3< j<g-l 
{q-l,q} ifj = q 

The last part of formulae (3) controls the global information accumulated during 
all the previous iterations and is represented by the value 

^j = f,{xj-xj.^y/''/{x"^n'^^, (5) 

where 

II = max{| f{xj) - f{xj-i) I /{xj - x^-i)^/^ : 2 < j < g}, 
jj^max ^ jnax{a;i - xt-i ■.2<i<q}. 



Step 3. For the intervals [xj-i,Xj],2 < j < q, compute their characteristics R{j) 
as follows 

R{j) = ri^jixj - xj.^y/'' + ZT'^l/N - i^j + ^i-i)' 2 < i < g, (6) 

riij{Xj Xj^i) 

where Zj = f{xj),l < j < q, and r > 1 is the reliability parameter of the 
method. 

Step 4. Calculate the new trial points a;*+-' G (0, 1), 1 < j < p, and their images 
y(a;'+-') G D,l < j < p, on the Peano curve m-approximation 

= 0.5(xt ,_i +xt.)- (2r)-i(| zt, - Zt,-i \ / fit^f sign{zt, - Zt^.,), (7) 



where 



ti = argniax{i?(i) : 1 < i < q}, 

tj = argniax{i?(i) : 1 < i < q,i tg,! < s < j — < j < p, 

p=p{l + l)<q{l)-l. 

Step 5. Execute trials of the (/ + l)th iteration at the points y{x'^~^^), 1 < j < p, 
where x''^^ , 1 < ,7 < P, are from (7). 

Step 6. If the stopping rule 

min{| xt, - xt,-i 1^/^: 1 < j < p} < e 

is not satisfied, then go to Step 1. Otherwise take the value 

z* = niin{2:j ■ I < i < q} 

as an estimate of the global minimum and corresponding to this value points 
y* G D as estimates of global minimizers and Stop. 

The algorithm described above belongs to the class of parallel characteristical 
methods introduced in [5] . In comparison with the other algorithms from the class 
[5] the main peculiarity of the new method consists in the following. Instead of 
usage the global Holder constant H or its estimates, PLT executes an adaptive 
local tuning on the behaviour of the reduced one-dimensional function f{x). The 
local tuning is based on the estimates /Xj of the local Holder constants Hj for every 
subinterval [ ], 2 < i < q{l), during every iteration I > \. 

The value fij from (3) represents the result of a balance between the global and 
local information accumulated during the search. If the interval [ small 
then, the local information (see (4)) represented by A^- is very important. The global 
information (see (5)) represented by 7^ is less important because the estimate /i 
of the global Holder constant (sec (2)) could be obtained at an interval being very 
far from [xj^i^Xj]. Vice versa, when the interval [a;j_i,a:j] is wide, then we cannot 
trust the local information from (3) and 7^ plays the most important part. The 
parameter ^ reflects our idea that f{x) is such that over every subinterval [xj-i,Xj] 
its Holder constant Hj > ^. 

Let us now consider convergence properties of the new method. Let {x'} be the 
sequence of the trial points generated by PLT during minimizing the function f{x). 
The first theorem on a level with other results asserts that only local minimizers of 
f{x) can be limit points of {x'}. 

Theorem 1. Let x e [0, 1] be a limit point of the sequence {a;^} and 

p{l) <Q <oo, l> 1. (8) 



Then the following results take place 



i. a X G (0; 1) then, there exist two subsequences of {x'^} such that the first one 
converges to x from the left and the second one - from the right; 

ii. the point S is a local minimizer of f{x) if the function f{x) has a finite number 
of local extrema; 

iii. if a part x there exists another limit point x, then f{x) = f{x); 

iv. for all g > 1 it follows = > /(x). 

Proof. Since PLT belongs to the class of parallel characteristical methods these 
results can be easily deduced from the general convergence theory presented in [5] . 
■ 

The second theorem describes sufficient convergence conditions of the sequence 
{a;'} to a global minimizer x*. This result can not be obtained within the general 

framework from [5] and we prove it. In order to proceed let us introduce {q} = 
{1, 2,3,.. .} as the sequence enumerating iterations executed by PLT. 

Theorem 2. If (8) is true and there exists an infinite subsequence {h}, {h} C {g}, 
such that for the interval [xj-i,Xj],j — G {/i}, containing the point a:* during 

the Zth PLT iteration the following inequality takes place 

r,ij > 2i-i/^i^ . + (4i-V^if2 _ ^2)1/2^ 
where 

Kj = max{(z,_i - f{x*)){x* - x^.,)-'"" , {z^ - f{x*)){x, - x*)-'^''}, (10) 



(9) 



M, =1 z,., - z, I {x, - (11) 

then, X* is a limit point of {a;*}. 

Proof. In order to start the proof we notice that the following inequality takes 
place for the estimates /Xj from (3): 

< ^ < fijii) < max{H, ^} < oo, 2 < j < q{l), I > 1. (12) 

Suppose now that there exists a limit point x' ^ x* of the trial sequence {x'^}. 
From (6), (12) and the first assertion of Theorem 1 we conclude for the interval 
[xi-i, Xi], i = containing x' at the Ith. iteration of PLT, that 

lim R{i{l)) = -4/(a;'). (13) 

Consider now the interval [xj-i,Xj],j = 

x*e[xj-uxj] (14) 

and suppose that x* is not a limit point of {x'^}. This means that there exists an 
iteration number m such that for alH > m 



a;3«+'= i {xj-x,x^\, j = l<k<p{l + 1), 



i.e. new trial points will not fall in the interval (14). Estimate now the characteristic 
R{j{l)),l > m, of this interval. It follows from (10) and (14) that 

zj-,-f{x*)<Kj{x*-Xj-i)^/'', 

z^-f{x*)<K^{x^-x*f"'. 
By summrizing these two inequalities and by using the designation 

a = {x* -Xj-i)/{xj -Xj-i), 
we obtain 

zj., + zj < 2/(x*) + Kj{{x* - a:,_i)i/^ + {xj - x*)V^) = 

= 2f{x*) + Kj{a^"' + (1 - a)i/^)(x,- - x^.^Y"" < 

< 2f{x*) + Kj{xj - a;j_i)i/^max{ai/^ + (1 - af'^ : < a < 1} = 

= 2(/(x*)+2-V^if^.(x,.-,c,_i)V^). 
From this estimate, (9), and (11) we have 

Rm)) = r^i^j - Xj-iY^'' + [Zj-i - zjf{rf,jr\xj - Xj-i)-'^'' - 2(^,-1 + zj) 

> (x, - x,.^f"'{r^l, + M]{r^i,)-^ - 2^-^/'' K^) - Af{x*) > -Af{x*) (15) 

for all iteration numbers I £ {h}. 

Since x* is a global minimizer and the sequence {h} is infinite, then from (13) 
and (15) it follows that an iteration number 1* will exist such that 

RWl) > RWl). (16) 

But, in according with the decision rules of PLT, this means that during the l*th. 
iteration one of p{l*) new trials will be executed at the interval (14). Thus, our 
assumption that x* is not a limit point of {a;^} is not true and theorem has been 
proved. ■ 

Corollary 1. Given the conditions of Theorem 2 all the limit points of the 
sequence {x"^} are the global minimizers of f{x). 
Proof. The corollary follows from the third assertion of Theorem 1. ■ 



Let X* be the set of the global minimizers of the function f{x). Corollary 1 
ensures that the set of limit points of the sequence {x'^} belongs to X*. Conditions 
ensuring coincidence of these sets are established by Corollary 2. 



Corollary 2. If condition (14) is fulfilled for all the points x* E X* , then the set 
of limit points of {x'^} coincides with X*. 

Proof. Again, the corollary is a straightforward consequence of Theorem 2 and 
the third assertion of Theorem 1. ■ 

Theorem 2 and its corollaries are very important both from theoretical and prac- 
tical viewpoints. It is known (see [2], [16], [17]) that every point y G D can have 
up to 2^ images on the curve. The global minimizer x* can have up to 2^ images 
on the curve too. To obtain an e-approximation of x* it is enough to find only one 
its image on the curve. 

We have proved that to have convergence to a global minimizer x* PLT needs 
the fulfilment of condition (9) (which is considerably weaker than the Lipseliitz 
condition) for one of the images of the point x* on the curve. Thus, (in contrast 
with the other methods from the class [5]) the new parallel method does not need 
the exact value of the precise Lipschitz constant L (neither its upper estimate) for 
the whole region D. It is enough that condition (9) is fulfilled in a neighbourhood 
of x* for one its image on the curve. In contrast with this, methods using in 
their work the exact Lipschitz (Holder) constant (or its upper estimate) will have 
convergence to all 2^ images on the curve. Of course, this fact leads to a significant 
slowing down the search and explains why PLT works faster. Results of numerecal 
experiments to be presented in the next Section confirm that the new approach 
permits to accelerate the search significantly. 

3. Numerical experiments 

In this section we compare performance of the new method PLT with the original 
information algorithm (lA), parallel information algorithm (PIA), and sequential 
information algorithm with local tuning (lALT). An ALLIANT FX/80 parallel 
mini-supercomputer having 4 processors has been used on the series of 100 two- 
dimensional multiextremal functions from [4] usually applied for testing information 
GO methods (see [4], [5], [12], [16], [18]): 



7 7 „ 7 7 



\ i=l j=l i=l i=l 

(17) 

where < < 1, < X2 < 1 and 

aij{x) = sin(i7ra;i) sin(j7ra;2), bij{x) = cos(i7ra;i) cos(j7rx2), 
and Aij,Bij,Cij,Dij are random coefficients from the interval [-1,1]. 



For all the methods in all the experiments we have used the 12-order approxima- 
tion of the Peano curve, initial points {0.2,0.4,0.6,0.9}, the reliability parameter 
r = 2.9 and the search accuracy e = 0.001. We have chosen ^ = 10~^ in PLT. 

In Tables 1,2 we present average results for 100 functions from the class (17). 
The column "%" shows a quantity of experiments in which global minima have 
been found. In Tab. 1 we compare the sequential method lALT and PLT. It is 
seen from the table that the introduced type of the parallelizm permits to achive 
high levels of speed up in comparison with the sequential method lALT. Note, that 
obtaining speed up higher than the number of processors used in the parallel case 
is possible due to parallel adaptive estimating the local Holder constants. 

In order to underline the effect obtained after introducing the local tuning, in 
Tab. 2 wc report the values of speed up obtained by using the parallel method 
PIA in comparison with the original sequential lA (see [18]). Both methods use an 
adaptive estimate of the global Holder constant in their work. 

In Tab. 3 we compare PLT working with the local Holder constants with PIA 
using the global Holder constant. The data from Tabs. 1,2 have been used. The 
obtained values of speed up both in time and in trials are shown. It is seen from 
Tab. 3 that in comparison with PIA the new method PLT functions faster more 
than 4.5 times in trials and more than 3.5 times in time. 



Table 1. Average results of the numerical experiments executed by PLT w.r.t. the sequential 
method lALT. 



Method 


Processors 


% 


Trials 


Time 


Speed up 


Speed up 












(trials) 


(time) 


lALT 


1 


98 


351.37 


11.15 






PLT 


2 


96 


339.00 


4.22 


1.97 


3.33 


PLT 


3 


98 


349.75 


2.73 


3.02 


7.15 


PLT 


4 


98 


348.24 


1.99 


3.94 


10.33 


Table 2. Average results of numerical experiments 


with the information algorithms 


using global 


estimates of the Holder constant. 










Method 


Processors 


% 


Trials 


Time 


Speed up 


Speed up 












(trials) 


(time) 


lA 


1 


100 


1575.12 


70.04 






PIA 


2 


100 


1596.08 


21.06 


1.97 


3.33 


PIA 


3 


100 


1562.61 


9.80 


3.02 


7.15 


PIA 


4 


100 


1599.92 


6.78 


3.94 


10.33 



Table 3. Speed up obtained by using PLT in compajrison with PIA on the data from Tabs. 1,2. 



Processors 


Speed up in trials 


Speed up in time 


2 


4.71 


4.99 


3 


4.47 


3.59 


4 


4.59 


3.41 



4. A brief conclusion 

In this paper a new parallel algorithm for solving global optimization multidimen- 
sional problems has been proposed. The method unifies two powerful approaches for 
accelerating the search : parallel computations and local tuning on the behavior of 
the objective function. Peano-type space-filling curves have been used to reduce the 
multidimensional problem to the one-dimensional one. For the obtained problem 
the new method adaptivcly estimates local Holder constants over different subintcr- 
vals of the one-dimensional search region to tune itself on the local behaviour of the 
reduced objective function. The local information is used by the method over the 
whole search region during the global search in contrast with traditional approaches 
usually starting a local search in a neighbourhood of a global minimizer after stop- 
ping the global procedure. It has been theoretically and numerically shown that 
the new technique permits to accelerate the search considerably. 
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